An Analysis of Document Text Fields and Prior Data in Web Information Retrieval

This report examines the combination of multiple sources of evidence such as content text, anchor text, Pagerank, and URL length data in order to increase the quality, or precision, of documents retrieved when using Web Information Retrieval. This report investigates what text fields (Body, Title, Anchor) taken from the structure of HTML documents and also from the hyperlink structure of the web perform best for the Adhoc, Homepage, and Topic Distillation tasks. It also outlines what Term Frequency normalisation technique gives the highest precision values for each text field and task combination. Complementary to this an investigation of merging techniques for text field retrieval sets is also conducted by comparing a linear combination of term frequencies to a linear combination of retrieval scores. Lastly, this report also covers in investigation into the use of prior data such as Pagerank and URL length within a DFR model.